Computers, tablets, smart phones, and other computing devices adopting natural user interfaces (NUIs) allowing users to input information as handwriting or drawings by writing directly on the screen or a digitizer are increasingly common. The process of receiving and rendering written or drawn information from a stylus is referred to as inking. The stylus may be a digital pen, mouse, finger or other suitable device that can be used to write or draw on the screen or digitizer. During inking, each move sequence of the stylus is recorded as a stroke storing the coordinates of and properties associated with the input. For example, a stroke may include the movements from the time when the pen tip is moved closed enough to be detected by the digitizer, the mouse button is depressed, or the finger touches the screen until the time that pin tip is moved away from the digitizer and no longer detectable, the mouse button is released, or the finger is no longer touching the screen. One or more strokes make up the ink (i.e., digital ink) used with applications running on the computing device.
On screen, the ink may appear as natural-looking handwriting or hand drawn pictures. In the case of handwriting, the ink may be converted to standard text through handwriting recognition. The converted text may be associated with the corresponding natural-looking handwriting as an alternate data format useful when working with (e.g., searching) inked information or may replace the natural-looking handwriting on screen and in the application (e.g., creating a typed document).
At times, the ink drawn on the screen severely lags behind the stylus. This lag is a result of the application attempting to do three actions simultaneously: process the ink input, render the ink to the screen, and save the ink to the application canvas (i.e., what the application displays to the user). Attempting to accomplish all of these tasks simultaneously slows each of them down because the user interface pipeline backs up quickly.
Lag is especially problematic for single-threaded applications, but remains a concern even for multi-threaded applications. In a multi-threaded application, lag may be addressed by creating additional threads and handling each action in a separate thread; however, this solution is not available for single-threaded applications and may not be viable or suitable for all multi-threaded applications.
When saving ink to the application canvas, lag may occur as a result of hardware timing constraints. For example, updates to the application canvas may be tied to the screen refresh frequency while input devices, such as the stylus, operate at the system bus speed. In a typical example, screens are refreshed on a 60 Hz cycle while the system bus operates at 133 Hz. Limited to one update of the application canvas per screen refresh (i.e., frame), the stylus generates ink more than twice as fast as it can be displayed by the application. As the user continues to write or draw, the ink lags further and further behind the stylus and is displayed to the user in choppy bursts rather than smoothly appearing with the movements of the stylus.
Different combinations of hardware and software have been found to produce significant lag where the ink does not appear for six or seven frames, which translates to the ink trailing the stylus by one to two inches. At a minimum, lag detracts from the pen and paper experience that inking is designed to provide. More than just being noticeable, lag creates efficiency and usability problems when the user has to stop and wait for the ink to catch up with the stylus so the user can see or make changes to what has been written or drawn. It is with respect to these and other considerations that the present invention has been made. Although relatively specific problems have been discussed, it should be understood that the embodiments disclosed herein should not be limited to solving the specific problems identified in the background.